Visual processing apparatus, visual processing method, program, recording medium, display device, and integrated circuit

ABSTRACT

It is possible to inhibit side effects even when an image that has sharp edge regions has been input, using a spatial processing portion ( 10 ) outputting surrounding image information US from an input image signal, a control signal generation portion ( 40 ) outputting an effect adjustment signal MOD according to a degree of flatness of an edge proximal region, and an effect adjustment portion ( 20 ) outputting a synthesized signal MUS that is synthesized by changing a ratio of the image signal IS and the surrounding image information US according to the effect adjustment signal MOD. Further, the side effects are inhibited using a visual processing portion ( 30 ) visually processing the image signal IS based on the synthesized signal MUS and the image signal IS.

This application is a Continuation of U.S. application Ser. No.12/159,077, filed Jun. 24, 2008, now U.S. Pat. No. 7,903,898 which is anational stage application of International application No.PCT/JP2007/057468, filed Apr. 3, 2007.

BACKGROUND OF THE INVENTION

1. Field of Invention

The present invention relates to visual processing apparatuses, visualprocessing methods, programs, recording media, display devices andintegrated circuits. In particular, the present invention relates tovisual processing apparatuses, visual processing methods, programs,recording media, display devices and integrated circuits, in which imageadjustment is performed, so as to vary an effect of visual processing ofan image.

2. Description of the Related Art

Conventionally, image processing devices that employ grayscaleprocessing and image processing devices that employ spatial processingare known for improving the quality of the image signal of an originalimage.

In grayscale processing, a lookup table (hereinafter, abbreviated as“LUT”) is used to effect pixel value conversion for each target pixel,regardless of the pixels surrounding that target pixel, and is alsoknown as gamma correction. For example, to enhance the contrast, pixelvalue conversion is performed using a LUT for enhancing grayscale levelsthat frequently appear in the original image. Examples of grayscaleprocessing using a LUT include grayscale processing in which a singleLUT is selected and used for an entire original image, and grayscaleprocessing in which a LUT is selected and used for each of a pluralnumber of image regions that are produced by partitioning the originalimage.

Spatial processing uses the value of a target pixel to be filtered andthe value of surrounding pixels in order to convert the value of thetarget pixel. Image signals that have been spatially processed are usedto enhance the contrast of the original image (for example, see PatentCitation 1).

-   Patent Citation 1: U.S. Pat. No. 4,667,304

Problem to be Solved by the Invention

One mode of processing to improve image quality to be close to humanvision is visual processing in which the value of a target pixel isconverted based on the contrast between the value of the target pixeland the values of pixels in the surrounding region. In such visualprocessing the brightness information is extracted from a wide regionaround the position of the target pixel in order to further bolster theprocessing effect.

However, since the value of the target pixel is determined from thecontrast between the value of the target pixel and the values of thesurrounding pixels, if there is a sharp edge region in the surroundingregion then the impact of the values of the surrounding pixels resultsin a visual processing output that changes gently near the edge, even inflat regions in which the pixel value fluctuates little. When a largechange in luminance occurs in a flat region, a shadow-like border occursin the region adjacent to the edge (hereinafter referred to as “sideeffect”) and results in an unnatural image.

BRIEF SUMMARY OF THE INVENTION

The present invention was arrived at in light of the above-mentionedissues, and it is an object of the invention to provide a visualprocessing apparatus, a visual processing method, a program, a recordingmedium, a display device, and an integrated circuit, that are capable ofsuppressing the side effect even when an image with sharp edge regionshas been input.

A first aspect of the invention is a visual processing apparatus that isprovided with a surrounding image information extraction portion thatextracts surrounding image information of an image signal that has beeninput, a visual processing portion that performs visual processing onthe image signal based on the image signal and the surrounding imageinformation, a control signal generation portion that outputs an effectadjustment signal for setting the effect of the visual processing, andan effect adjustment portion that sets the effect of the visualprocessing according to the effect adjustment signal. The control signalgeneration portion determines edge proximal regions and flat regionsthat are present in an image region of the image that is formed by theimage signal, and generates an effect adjustment signal that realizesthe visual processing based on the image signal and the surroundingimage information, on those image regions that have been determined notto be an edge proximal region or a flat region, whereas it generates aneffect adjustment signal that realizes visual processing with a weakereffect than the effect of the visual processing based on the imagesignal and the surrounding image information, on those image regionsthat have been determined to be an edge proximal region as well as aflat region.

With this visual processing apparatus, the control signal generationportion outputs an effect adjustment signal for setting the effect ofvisual processing, the surrounding image information extraction portionextracts surrounding image information of the image signal that has beeninput, and the effect adjustment portion sets the effect of the visualprocessing according to the effect adjustment signal, and thus theeffect of visual processing can be varied according to the effectadjustment signal, and side effects can be inhibited by adjusting thevisual processing effect in image regions where side effects occur.Further, with this visual processing apparatus, the control signalgeneration portion determines edge proximal regions and flat regionsthat are present in an image region of the image that is formed by theimage signal, and generates an effect adjustment signal that realizesthe visual processing based on the image signal and the surroundingimage information on those image regions that have been determined notto be an edge proximal region or a flat region, whereas it generates aneffect adjustment signal that realizes visual processing with a weakereffect than the effect of the visual processing based on the imagesignal and the surrounding image information on those image regions thathave been determined to be an edge proximal region as well as a flatregion, and thus the effect of visual processing on image regions inwhich side effects readily occur is weakened, making it possible toinhibit the occurrence of side effects. In particular, the side effectin which a so-called halo occurs in an image can be efficientlyinhibited.

It should be noted that here, “visual processing with a weak effect” isa concept that also includes not generating a visual processing effectat all, that is, not performing visual processing. Further, “surroundingimage information” refers to information that is drawn from the targetpixels and the pixels surrounding the target pixels, and for example isthe mean brightness (grayscale value) of a region of N×N pixels with thesurrounding pixels at the center. It should be that in order to obtainsurrounding image information it is not always necessary to performprocessing in pixel units, and it is also possible to obtain surroundingimage information by performing processing in block units, each made ofa plurality of pixels.

A second aspect of the invention is the first aspect of the invention,in which the visual processing portion performs visual processing foradjusting the local contrast on the image signal based on the imagesignal and the surrounding image information.

Thus, the side effects can be inhibited even in a visual processingapparatus that realizes visual processing in which the local contrast isadjusted.

It should be noted that here, “local contrast” refers to the contrastthat is gained from comparison of the brightness between the targetpixel and the pixels surrounding it, and “visual processing foradjusting the local contrast” for example refers to processing in whichthe contrast of a local image region is adjusted based on the brightnessratio between the target pixel and the surrounding pixels in a localimage region. It should be noted that it goes without saying that it ispossible to perform processing in block units as well as in pixel units.Further, “visual processing for adjusting the local contrast” is aconcept that includes processing that is based on a contrast in colorinformation (such as luminance, vividness, and hue), as well as thebrightness contrast.

A third aspect of the invention is the first aspect or second aspect ofthe invention, in which the effect adjustment signal generation portionoutputs the effect adjustment signal according to an amount of change inthe surrounding image information.

Thus, it is possible to inhibit the side effects that occur inconjunction with a change in the surrounding image information.

A fourth aspect of the invention is the first aspect or second aspect ofthe invention, in which the control signal generation portion includes achange amount detection portion for detecting an amount of change in thesurrounding image information, and an effect adjustment signalgeneration portion that generates the effect adjustment signal based onthe amount of change that has been detected by the change amountdetection portion.

Thus, it is possible to inhibit the side effects that occur inconjunction with the amount of change in the surrounding imageinformation.

A fifth aspect of the invention is the first aspect or second aspect ofthe invention, in which the control signal generation portion includes aflatness detection portion that detects a degree of flatness of flatregions, whose difference in luminance with an adjacent region is at orbelow a predetermined value, from the image signal, an edge detectionportion that detects an edge amount of an edge region, whose differencein luminance with an adjacent region is at or above a predeterminedvalue, from the image signal, an edge proximity detection portion forcalculating a degree of edge proximity that indicates the degree towhich an image region is near an edge, from the edge amount, and aneffect adjustment signal generation portion that generates the effectadjustment signal based on the degree of flatness that has been detectedby the flatness detection portion and the degree of edge proximity thathas been calculated by the edge proximity detection portion.

Thus, the side effects in flat regions that are near an edge region canbe inhibited even when an image that has sharp edge regions is input.

A sixth aspect of the invention is any one of the first aspect throughthe fifth aspect of the invention, in which the effect adjustmentportion outputs a first synthesized signal that is obtained bysynthesizing the image signal and the surrounding image informationaccording to the effect adjustment signal, and the visual processingportion performs visual processing on the image signal based on thefirst synthesized signal and the image signal.

Thus, additionally, the visual processing portion can select differentgrayscale conversion processing based on the first synthesized signal,and can visually process the image signal based on the grayscaleconversion processing that has been selected, and thus can vary (adjust)the effect of the visual processing.

A seventh aspect of the invention is any one of the first aspect throughthe fifth aspect of the invention, in which the effect adjustmentportion outputs a second synthesized signal that is obtained bysynthesizing the image signal and the output that has been visuallyprocessed by the visual processing portion according to the effectadjustment signal.

Thus, additionally, it is possible to perform the output after changingthe ratio of the image signal and the processed signal in accordancewith the effect adjustment signal, and this allows the effect of visualprocessing to be varied (adjusted).

An eighth aspect of the invention is a visual processing method thatincludes a surrounding image information extraction step of extractingsurrounding image information of an image signal that has been input, avisual processing step of performing visual processing on the imagesignal based on the image signal and the surrounding image information,a control signal generation step of outputting an effect adjustmentsignal for setting the effect of the visual processing, and an effectadjustment step of setting the effect of the visual processing accordingto the effect adjustment signal. In the control signal generation step,edge proximal regions and flat regions that are present in an imageregion of the image that is formed by the image signal are determined,and an effect adjustment signal that realizes the visual processingbased on the image signal and the surrounding image information on thoseimage regions that have been determined not to be an edge proximalregion or a flat region, is generated, whereas an effect adjustmentsignal that realizes visual processing with a weaker effect than theeffect of the visual processing based on the image signal and thesurrounding image information on those image regions that have beendetermined to be an edge proximal region as well as a flat region, isgenerated.

Thus, it is possible to achieve a visual processing method that attainsthe same effects as the first aspect of the invention.

A ninth aspect of the invention is a program for causing a computer toexecute a surrounding image information extraction step of extractingsurrounding image information of an image signal that has been input, avisual processing step of performing visual processing on the imagesignal based on the image signal and the surrounding image information,a control signal generation step of outputting an effect adjustmentsignal for setting the effect of the visual processing, and an effectadjustment step of setting the effect of the visual processing accordingto the effect adjustment signal. In the control signal generation step,edge proximal regions and flat regions that are present in an imageregion of the image that is formed by the image signal are determined,and an effect adjustment signal that realizes the visual processingbased on the image signal and the surrounding image information on thoseimage regions that have been determined not to be an edge proximalregion or a flat region, is generated, whereas an effect adjustmentsignal that realizes visual processing with a weaker effect than theeffect of the visual processing based on the image signal and thesurrounding image information on those image regions that have beendetermined to be an edge proximal region as well as a flat region, isgenerated.

Thus, it is possible to achieve a program that attains the same effectsas the first aspect of the invention.

A tenth aspect of the invention is a non-transitory computer-readablerecording (storage) medium having a program recorded/stored thereon forcausing a computer to execute a surrounding image information extractionstep of extracting surrounding image information of an image signal thathas been input, a visual processing step of performing visual processingon the image signal based on the image signal and the surrounding imageinformation, a control signal generation step of outputting an effectadjustment signal for setting the effect of the visual processing, andan effect adjustment step of setting the effect of the visual processingaccording to the effect adjustment signal. In the control signalgeneration step, edge proximal regions and flat regions that are presentin an image region of the image that is formed by the image signal aredetermined, and an effect adjustment signal that realizes the visualprocessing based on the image signal and the surrounding imageinformation on those image regions that have been determined not to bean edge proximal region or a flat region, is generated, whereas aneffect adjustment signal that realizes visual processing with a weakereffect than the effect of the visual processing based on the imagesignal and the surrounding image information on those image regions thathave been determined to be an edge proximal region as well as a flatregion, is generated.

Thus, it is possible to achieve a recording medium that attains the sameeffects as the first aspect of the invention.

An eleventh aspect of the invention is an integrated circuit thatincludes a surrounding image information extraction portion thatextracts surrounding image information of an image signal that has beeninput, a visual processing portion that performs visual processing onthe image signal based on the image signal and the surrounding imageinformation, a control signal generation portion that outputs an effectadjustment signal for setting the effect of the visual processing, andan effect adjustment portion that sets the effect of the visualprocessing according to the effect adjustment signal. The control signalgeneration portion determines edge proximal regions and flat regionsthat are present in an image region of the image that is formed by theimage signal, and generates an effect adjustment signal that realizesthe visual processing based on the image signal and the surroundingimage information, on those image regions that have been determined notto be an edge proximal region or a flat region, whereas it generates aneffect adjustment signal that realizes visual processing with a weakereffect than the effect of the visual processing based on the imagesignal and the surrounding image information, on those image regionsthat have been determined to be an edge proximal region as well as aflat region.

Thus, it is possible to achieve an integrated circuit that attains thesame effects as the first aspect of the invention.

A twelfth aspect of the invention is a display device that includes adata reception portion that receives image data that have beentransmitted or broadcast, a decoding portion that decodes the image datathat have been received into video data, the visual processing apparatusaccording to any one of the first through seventh aspects of theinvention for visually processing the decoded video data and outputtingan output signal, and a display portion that performs a display of theoutput signal that has been visually processed by the visual processingapparatus.

Thus, it is possible to achieve visual processing that attains the sameeffects as in the first through seventh aspects of the invention on adisplay device.

With the invention, it is possible to provide a visual processingapparatus, a visual processing method, a program, a recording medium,and an integrated circuit that can inhibit side effects even when animage that has sharp edge regions has been input.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing the configuration of a visualprocessing apparatus of the first embodiment of the invention.

FIG. 2 is an explanatory diagram for describing the two-dimensionalgrayscale characteristics of the same.

FIGS. 3A and 3B describe the output of the processed signal OS of thesame.

FIG. 4 is a block diagram that shows the configuration of the controlsignal generation portion of the same.

FIGS. 5A-5E describe the output of the effect adjustment signal of thesame.

FIG. 6 is a flowchart that describes the operation of the visualprocessing apparatus of the same.

FIGS. 7A and 7B show the configuration of the control signal generationportion of a modified example of the same.

FIG. 8 is an explanatory diagram for describing the effect adjustmentsignal of a modified example of the same.

FIG. 9 is a block diagram that shows the configuration of a visualprocessing apparatus according to a second embodiment of the invention.

FIG. 10 is a block diagram that shows the configuration of a visualprocessing apparatus according to a third embodiment of the invention.

FIG. 11 is a block diagram that shows the configuration of a visualprocessing system according to a fourth embodiment of the invention.

FIG. 12 is an explanatory diagram for describing the two-dimensionalgain characteristics of the same.

FIGS. 13A-13E describe the output of the effect adjustment signal of thevisual processing apparatus according to the first embodiment of theinvention.

FIGS. 14A-14D describe the output of the effect adjustment signal of thevisual processing apparatus according to the first embodiment of theinvention.

FIG. 15 is a block diagram that shows the configuration of a computeraccording to an embodiment of the invention.

FIG. 16 is a diagram of the overall configuration of the content supplysystem according to the fifth embodiment of the invention.

FIG. 17 is a front view of a portable telephone that is provided withthe visual processing apparatus according to the fifth embodiment.

FIG. 18 is a block diagram for describing the overall configuration ofthe portable telephone according to the fifth embodiment.

FIG. 19 is an explanatory diagram of the overall configuration of thedigital broadcast system according to the fifth embodiment.

EXPLANATION OF REFERENCE

-   1, 2, 3 visual processing apparatus-   4 gain-type visual processing system-   5 gain-type visual processing apparatus-   10 spatial processing portion-   11 multiplier-   12 divider-   20, 21, 22 effect adjustment portion-   30, 31, 32 visual processing portion-   40, 70 control signal generation portion-   41 edge detection portion-   42 edge proximity detection portion-   43 flatness detection portion-   44, 72 effect adjustment signal generation portion-   60 profile A-   61 profile B-   71 change amount detection portion-   100 CPU-   101 ROM-   102 RAM-   103 input portion-   104 output portion-   105 memory portion-   106 communications portion-   107 drive-   108 disk-   109 memory card-   110 bus-   111 network

DETAILED DESCRIPTION OF THE INVENTION

Below, embodiments of the invention are described in detail withreference to the drawings.

First Embodiment

In general, natural images have many gradations, and by performingvisual processing on natural images it is possible to obtain sharpimages with high local contrast, for example. On the other hand, whenvisual processing is performed on an image with steep edges, the sideeffect is easily stands out. When the strength of the visual processingis reduced in order to suppress this side effect, the processing becomesweak even for natural images and the resulting image is not sharp.

Thus, by toning down visual processing only near edges, it is possibleto maintain the processing effect for the natural image overall whilesuppressing the side effect near edges.

The visual processing apparatus of the first embodiment of the inventionperforms adjustment (of the strength or correction amount) by outputtingan effect adjustment signal for varying the effect of the visualprocessing in order to vary the effect of visual processing according tothe effect adjustment signal.

It also performs adjustment by detecting regions adjacent to an edge orflat regions adjacent to an edge in the image targeted for visualprocessing, and creates an effect adjustment signal from the edge amountand the degree of flatness and varies the effect of visual processing inaccordance with the effect adjustment signal.

Thus, even when an image with a steep edge region has been input to thevisual processing apparatus, it is possible to obtain the visualprocessing effect while suppressing the side effect near edges.

Here, visual processing is processing for giving characteristics thatare close to human vision, and determines the value of an output signalbased on the contrast between the value of a target pixel of the imagesignal that has been input and the values (brightness) of pixelssurrounding that target pixel. The visual processing may be adopted forbacklight correction, knee processing, D range compression, colorprocessing, and brightness adjustment (including grayscale processingand contrast adjustment), for example.

In this embodiment of the invention, the luminance component Y or thebrightness component L of a YCbCr color space, a YUV color space, a Labcolor space, a Luv color space, a YIQ color space, or a YPbPr colorspace is defined as the luminance signal. Hereinafter, the luminancesignal is described as the image signal.

The visual processing apparatus of the first embodiment of the inventionis described using FIGS. 1 through 8. FIG. 1 is a block diagram showingthe configuration of a visual processing apparatus 1 according to thefirst embodiment of the invention.

In FIG. 1, the visual processing apparatus 1 according to the firstembodiment of the invention is provided with a spatial processingportion 10 for outputting surrounding image information (unsharp signal)US from an image signal that has been input, a control signal generationportion 40 for outputting an effect adjustment signal MOD in accordancewith the degree of flatness of the edge proximal region, an effectadjustment portion 20 for outputting a synthesized signal MUS that issynthesized changing the proportion of the image signal IS and thesurrounding image information US according to the effect adjustmentsignal MOD, and a visual processing portion 30 for visually processingthe image signal IS based on the synthesized signal MUS and the imagesignal IS.

The various functional sections of the visual processing apparatus 1 aredescribed below.

The spatial processing portion 10 extracts the value of a target pixeland the values of pixels in the region around the target pixel(hereinafter, called “surrounding pixels”) from the image signal IS, anduses the values of the extracted pixels to filter the image signal IS.

For example, the spatial processing portion 10 performs a low-passfilter on the image signal IS to create an unsharp signal US. Theunsharp signal US is created by a computation such as the following.US=(Σ[W _(ij) ]×[A _(ij)])÷(Σ[W _(ij)])

Here, [W_(ij)] is the weight coefficient of the pixel located at thei-th row j-th column, of the target pixel and the surrounding pixels,and [A_(ij)] is the value of the pixel located at the i-th row j-thcolumn, of the target pixel and the surrounding pixels. The symbol Σmeans to take the sum of the target pixel and the surrounding pixels.

It should be noted that it is possible to assign a weight coefficientwith a smaller value the larger the absolute value of the differencebetween pixel values, and it is also possible to assign a smaller weightcoefficient the larger the distance from the target pixel. The region ofthe surrounding pixels is a size that is set in advance in accordancewith the effect, and the visual effect can be increased by setting thisregion to larger than a predetermined size. For example, if the size ofthe target image is 1024 pixels vertically by 768 pixels horizontally,then by creating an unsharp signal US from a region that is at least 80pixels vertically by 80 pixels horizontally, the visual effect can beincreased compared to local regions of about 3 pixels vertically by 3pixels horizontally.

A FIR (Finite Impulse Response)-type low-pass spatial filter or an IIR(Infinite Impulse Response)-type low-pass spatial filter, which arecommonly used to create unsharp signals, can be used as the low-passspatial filter.

Next, the effect adjustment portion 20 synthesizes the image signal ISand the unsharp signal US by interpolation in accordance with the effectadjustment signal MOD that has been output from the control signalgeneration portion 40, and outputs a synthesized signal MUS. Thesynthesized signal MUS is obtained by an interpolation computation suchas that of Eq. 1 according to the effect adjustment signal MOD. Thecontrol signal generation section 40 is discussed later.MUS=US×MOD+IS×(1.0−MOD)  Eq. 1

Here, the value of the effect adjustment signal MOD is changed withinthe range of 0.0 to 1.0, with no effect when the value of the effectadjustment signal MOD is 0.0 and a maximum processing strength when thevalue of the effect adjustment signal MOD is 1.0. It should be notedthat Eq. 1 can be modified as in Eq. 2, and the synthesized signal MUSsimilarly can be created.MUS=(US−IS)×MOD+IS  Eq. 2

Next, the visual processing portion 30 converts the grayscale of theimage signal IS according to the synthesized signal MUS from the effectadjustment portion 20.

The visual processing portion 30 for example performs grayscaleconversion based on the two-dimensional grayscale conversioncharacteristics shown in FIG. 2. Here, two-dimensional grayscaleconversion refers to grayscale conversion in which the value of anoutput is determined for the two inputs, that is, the synthesized signalMUS and the image signal IS. Based on the two-dimensional grayscaleconversion characteristics, the visual processing portion 30 outputs aprocessed signal OS for the image signal IS and the synthesized signalMUS. Various visual effects can be produced from the grayscaleconversion characteristics.

The two-dimensional grayscale conversion characteristics are describedusing FIG. 2. FIG. 2 is an explanatory diagram for describing thetwo-dimensional grayscale conversion characteristics. Here, thehorizontal axis is the image signal IS that has been input, and thevertical axis is the output of the converted processed signal OS.

As shown in FIG. 2, the two-dimensional grayscale conversion haspredetermined grayscale conversion characteristics that match the signallevel of the synthesized signals MUS0 to MUSn. That is, two-dimensionalgrayscale conversion is achieved by converting the input signal IS (thegrayscale value of IS) into a processed signal OS (the grayscale valueof OS) with a grayscale conversion curve that has been selected fromamong the grayscale conversion curves MUS0 to MUSn based on the signallevel (grayscale value) of the synthesized signal MUS. For example, thegrayscale conversion curve MUS1 of FIG. 2 is selected when the MUSsignal level (grayscale value) is 1, and the grayscale conversion curveMUS120 is selected when the MUS signal level (grayscale value) is 120.However, it is not absolutely necessary to prepare the same number ofgrayscale conversion curves MUS0 to MUSn as the number of grayscalevalues of the MUS signal, and for example it is also possible to preparea smaller number of grayscale conversion curves MUS0 to MUSn than thenumber of grayscale values of the MUS signal, and with regard to thegrayscale conversion curves that have not been readied, to achievetwo-dimensional grayscale conversion by calculating a grayscaleconversion curve that corresponds to the grayscale value of the MUSsignal by interpolation from the grayscale conversion curves that havebeen prepared.

In the two-dimensional grayscale conversion, when the pixel value of theimage signal IS is an 8-bit value, the pixel value of the output signalOS that corresponds to the value of the image signal IS separated into256 levels is determined based on the predetermined two-dimensionalgrayscale conversion characteristics. The grayscale conversioncharacteristics are grayscale conversion curves that have predeterminedgamma conversion characteristics, and the relationship is such that theoutput monotonically decreases along with the subscript of thesynthesized signal MUS. It should be noted that even if there are placeswhere the output partially does not monotonically decrease along withthe subscript of the synthesized signal MUS, it is sufficient for it tobe substantially monotonically decreasing. As shown in FIG. 2, in thetwo-dimensional grayscale conversion characteristics, the relationshipof (the output value when MUS=MUS0)≧(the output value when MUS=MUS1)≧ .. . ≧(the output value when MUS=MUSn) is satisfied for the brightnessvalues of all image signal IS pixels.

With the two-dimensional grayscale conversion characteristics shown inFIG. 2, for an input image signal IS with a value “a,” the visualprocessing portion 30 selects MUS0 when the brightness value of thesurrounding region is small so as to obtain a value of “P” for theprocessed signal OS, whereas it selects MUSn when the brightness valueof the surrounding region is large so as to obtain a value of “Q” forthe processed signal OS. In this way, even when the input image signalIS has the value “a,” the processed signal OS can be significantlyvaried between the value “P” to the value “Q” depending the change inthe brightness value of the surrounding region. By doing this, thecontrast of dark areas can be enhanced in accordance with thesynthesized signal MUS.

On the other hand, if the synthesized signal MUS is set equal to theimage signal IS so as to eliminate the effect of visual processing, thenit is possible to have the grayscale conversion characteristics of curve2 shown in FIG. 2. With the grayscale conversion characteristics ofcurve 2, brightness adjustment (gamma conversion) of the entire image ispossible, but there is no visual effect such as an increase in thecontrast only in local dark area regions.

It should be noted that it is possible to produce various visualprocessing effects by changing the two-dimensional grayscale conversioncharacteristics, and thus the processing can be adopted for kneeprocessing, DR compression, color processing, or brightness adjustment(including grayscale processing and contrast adjustment), for example.

Next, in the visual processing portion 30, the processed signal OS whenthe effect of visual processing has been changed based on thesynthesized signal MUS is described using FIGS. 3( a) and 3(b). FIG. 3is an explanatory diagram for describing the processed signal OS output.

In FIG. 3( a), the horizontal axis is the pixel position to beprocessed, and the vertical axis is the output of the synthesized signalMUS.

For example, when the value of the effect adjustment signal MOD has beenset to 0.5, the synthesized signal MUS becomes an output that isintermediate between the image signal IS and the unsharp signal US.

At this time, as shown in FIG. 3( b), if OS(IS,IS) is a processed signalOS that has been visually processed based on only the image signal ISand OS(IS,US) is a processed signal OS that has been visually processedbased on the image signal IS and the unsharp signal US, then theprocessed signal OS(IS,MUS) that is obtained by visual processingaccording to the image signal IS and the synthesized signal MUS is anoutput that is intermediate between OS(IS,IS) and OS(IS,US).

Thus, the synthesized signal MUS is equal to US when the value of theeffect adjustment signal MOD is 1.0, and a processed signal OS(IS,US) inwhich there is a maximum visual processing effect is output. On theother hand, the synthesized signal MUS is equal to IS when the value ofthe effect adjustment signal MOD is 0.0, and a processed signalOS(IS,IS) in which there is no visual processing effect is output.

In this way, the visual processing portion 30 can strengthen or weakenthe effect of visual processing of the dark area contrast based on thesynthesized signal MUS. Thus, the visual processing apparatus 1 canachieve various visual effects with varying effects, from the effect ofprocessing in which only the brightness of the overall image istransformed, to the effect of processing in which the contrast of alocal region is varied (changed) with the surrounding brightness.

It should be noted that the visual processing apparatus 1 can achieveknee processing, DR compression processing, and color processing, forexample, by changing the two-dimensional grayscale conversioncharacteristics.

It is also possible for the visual processing portion 30 to have atwo-dimensional lookup table (hereinafter, referred to as “2D LUT”).Grayscale conversion is performed by setting characteristic data(hereinafter, referred to as the “profile”) shown in FIG. 2 in the 2DLUT of the visual processing portion 30.

The visual processing portion 30 can also perform visual processing withan arithmetic circuit. In particular, if profiles, which arecharacteristics that can be approximated by a simple straight line, areset in the 2D LUT of the visual processing portion 30, then it ispossible to eliminate the table of the 2D LUT and reduce the circuitscale of the visual processing apparatus 1.

Next, the control signal generation portion 40 is described using FIGS.4 and 5( a)-5(e). FIG. 4 is a block diagram showing the configuration ofthe control signal generation portion 40, and FIG. 5( a)-5(e) are anexplanatory diagram for describing the output of the effect adjustmentsignal MOD.

As shown in FIG. 4, the control signal generation portion 40 is providedwith an edge detection portion 41 for detecting the edge amount, thatis, the luminance difference of each adjacent region, from the imagesignal IS, an edge proximity detection portion 42 for detecting thedegree of proximity of the edge region from the edge amount, a flatnessdetection portion 43 for detecting the degree of flatness of flatregions whose luminance difference with an adjacent region is at orbelow a predetermined value, and an effect adjustment signal generationportion 44 for outputting an effect adjustment signal MOD in accordancewith the degree of edge proximity and the degree of flatness.

The edge detection portion 41 detects the edge amount from the imagesignal IS for each region of a predetermined range. The edge detectionportion 41 detects the edge amount EG using an edge detection filter(not shown) such as a first derivative filter like a Sobel filter or aPrewitt filter or a second derivative filter like a Laplacian filter.The edge detection portion 41 for example outputs an edge amount likethat shown in FIG. 5( b) when the image signal IS shown in FIG. 5( a)has been input. Here, in FIG. 5( a) the vertical axis is the value ofthe image signal IS, and the horizontal axis is the pixel position ofthe pixel being processed. The vertical axis in FIG. 5( b) is the edgeamount EG, and the horizontal axis is the pixel position of the pixelbeing processed.

The edge proximity detection portion 42 detects regions near an edge.For example, the edge proximity detection portion 42 applies a low-passfilter on the edge amount that is detected for each predeterminedregion, and outputs a degree of proximity that becomes a larger outputas the edge proximity increases. The degree of edge proximity may alsobe set by spatially shifting the edge amount EG horizontally orvertically and adding or averaging these. It is also possible for thedegree of edge proximity to be the edge amount EG processed by a MAXfilter. For example, as shown in FIG. 5( c), the edge proximitydetection portion 42 outputs a degree of edge proximity that becomes alarger output as the edge proximity increases. Here, the vertical axisin FIG. 5( c) is the degree of edge proximity, and the horizontal axisis the pixel position of the pixel being processed.

To determine whether a region is flat, the flatness detection portion 43preferably calculates the flatness FT using the luminance difference ina wider range than the edge detection portion 41. This is because peoplecan recognize side effects, such as the feeling that the image is notnatural, when looking at an image looking for flat regions that have arelatively wide area. For example, when viewing a spatially-processedimage on a high vision (HDTV) display device at the optimal viewingdistance of 3H (H is the height of the screen), the flatness FTpreferably is detected in a range that is at least H/20 (thiscorresponds to 54×54 pixels in the case of full HD (full spec highvision) (1920×1080 pixels). However, the increasing size of screens hasled to a shorter relative viewing distance, and thus it may also bepossible to detect the flatness FT in a range of at least H/30(similarly, this corresponds to 36×36 pixels). At least some effect isobtained by detecting the flatness FT in a range of H/50 or more(similarly, this corresponds to 22×22 pixels).

The flatness detection portion 43 detects the flatness degree (flatnessFT) of flat regions in which the difference in luminance with anadjacent region is at or below a threshold value. For example, the areaof regions in which the luminance difference with the adjacent region isat or below a predetermined value is found, and a larger flatness degree(flatness FT) is output the larger the area.

As shown in FIG. 5( d), the flatness detection portion 43 detects thedifference in luminance from the adjacent region from the output of theedge amount of FIG. 5( b), and outputs a larger flatness degree(flatness FT) the smaller the luminance difference. The vertical axis inFIG. 5( d) is the flatness FT, and the horizontal axis is the pixelposition of the pixel being processed.

As shown in FIG. 5( e), the effect adjustment signal generation portion44 hybridizes the degree of proximity of FIG. 5( c) and the degree offlatness of FIG. 5( d), and outputs an effect adjustment signal MOD thatweakens the visual effect the larger the edge proximity degree and thehigher the degree of flatness. Here, the vertical axis in FIG. 5( d) isthe output of the effect adjustment signal MOD, and the horizontal axisis the pixel position of the pixel being processed. The visual effect bythe visual processing apparatus 1 becomes stronger the larger the valueof the effect adjustment signal MOD.

By doing this, the effect adjustment signal generation portion 44, asshown in FIG. 5( e), creates an output that weakens the visual effect inregions that are near an edge, and creates an output that strengths thevisual effect in regions that are removed from regions that are near anedge. Further, in regions near an edge, based on the degree of flatnessthe effect adjustment signal generation portion 44 creates an outputthat weakens the visual effect the larger the degree of flatness, andcreates an output that strengthens the visual effect the smaller thedegree of flatness.

FIG. 13( a) shows the relationship between the pixel position and theimage signal IS, FIG. 13( b) shows the relationship between the pixelposition and the edge amount EG, FIG. 13( c) shows the relationshipbetween the pixel position and the edge proximity, FIG. 13( d) shows therelationship between the pixel position and the flatness FT, and FIG.13( e) shows the relationship between the pixel position and the effectadjustment signal MOD. It should be noted that FIG. 13( b) shows theedge amount EG, FIG. 13( c) shows the edge proximity, FIG. 13( d) showsthe flatness FT, and FIG. 13( e) shows the effect adjustment signal MOD,when the image signal IS shown in FIG. 13( a) has been input. Here, theregion indicated by EF in FIG. 13 is the flatness area near the edge.The effect adjustment signal MOD is smaller in the flat region EF nearthe edge.

Thus, with the visual processing apparatus 1, it is possible to achievevisual processing with which the side effect can be reduced only nearedges, and which has an excellent visual processing effect for naturalimages.

Next, the operation of the visual processing apparatus 1 is describedusing FIG. 6. FIG. 6 is a flowchart for describing the operation of thevisual processing apparatus 1.

As shown in FIG. 8, an image is input to the visual processing apparatus1 (S101), and the edge detection portion 41 detects an edge amount,which is the difference in luminance, for each adjacent region from theimage signal IS (S102).

Next, the edge proximity detection portion 42 of the visual processingapparatus 1 processes the edge amounts with a low-pass filter to detecta degree of proximity from the edge amounts (S103). The flatnessdetection portion 43 of the visual processing apparatus 1 then detectsluminance difference from the edge amount to detect the degree offlatness near the edges (S104).

Next, the effect adjustment signal generation portion 44 of the visualprocessing apparatus 1 combines the proximity degree that has beenoutput from the edge proximity detection portion 42 and the flatnessdegree that has been output from the flatness detection portion 43 tocreate an effect adjustment signal MOD (S105).

Next, the visual processing portion 20 of the visual processingapparatus 1 creates a synthesized signal MUS that is synthesizedchanging the ratio of the image signal IS and the unsharp signal US inaccordance with the effect adjustment signal MOD (S106).

Next, the visual processing portion 30 of the visual processingapparatus 1 selects a single curve from the two-dimensional grayscaleconversion characteristics shown in FIG. 2 based on the synthesizedsignal MUS, and converts the image signal IS (S107). By doing this, thevisual processing apparatus 1 executes visual processing that has beenadjusted so that the effect of visual processing is altered according tothe synthesized signal MUS.

Next, the visual processing apparatus 1 determines whether or not thereare pixels to process next (S108). If there are no more pixels thatrequire processing, then visual processing is completed. On the otherhand, if there are further pixels that require processing, then theprocedure is returned to step S101 and the next image (pixel) is input.Thereafter, the steps from S101 to S108 are repeated until there are nolonger any pixels that require processing.

Thus, with the visual processing apparatus 1 of the first embodiment ofthe invention, it is possible to achieve visual processing that allowsthe side effect to be reduced only near edges, and which has anexcellent visual processing effect for natural images.

It should be noted that although the visual processing apparatus 1 findsthe degree of edge proximity from the edge amount and finds the degreeof flatness from the input image signal IS, and creates the effectadjustment signal MOD based on the degree of edge proximity and thedegree of flatness, it is also possible to create the effect adjustmentsignal MOD from the amount of change in the unsharp signal US by thespatial processing portion 10.

Below is described a method for detecting a flat region near an edgeaccording to a modified example of the control signal generation portion40.

An implementation in which the effect adjustment signal MOD is createdfrom the amount of change in the unsharp signal US is described usingFIGS. 7( a), 7(b) and 14(a)-14(d). FIG. 7( a) is a block diagram showingthe configuration of the visual processing apparatus 1 according to thismodified example of the embodiment. That is, it shows a modified exampleof the visual processing apparatus 1 that is shown in FIG. 1. Thisvisual processing apparatus differs from the visual processing apparatus1 of FIG. 1 in that the input of a control signal generation portion 70(a modified example of the control signal generation portion 40) is notthe input signal IS but rather an unsharp signal US signal. FIG. 7( b)is a block diagram that shows the configuration of the control signalgeneration portion 70, which is a modified example of the control signalgeneration portion 40.

As shown in FIG. 7( b), the control signal generation portion 70 isprovided with a change amount detection portion 71 for detecting theamount of change in the unsharp signal US, and an effect adjustmentsignal generation portion 72 for outputting an effect adjustment signalMOD in accordance with the amount of change that has been detected.

The unsharp signal US is obtained by cutting out (reducing) thehigh-frequency signal component of a natural image, but leaves steepedge components. Thus, by extracting regions near an edge based on edgecomponents that remain in the unsharp signal US, it is possible toapproximately detect flat portions near edges.

In this embodiment, the input of the control signal generation portion70 is set as the unsharp signal US, and flat sections near edges aredetected by calculating the amount of change in that signal. Thus, inthe flatness detection, in which it is preferable that a large area(image region) is referenced, it is possible to reduce the image regionthat is referenced and thus reduce the amount of processing required todetect a flat section.

The change amount detection portion 71 performs this detection using anedge detection filter (not shown) such as a first derivative filter likea Sobel filter or a Prewitt filter or a second derivative filter like aLaplacian filter.

This may be combined with a low-pass filter or a MAX filter in order toadjust the width of the regions near an edge.

For example, when an image signal IS like that in FIG. 14( a) has beeninput, resulting in an unsharp signal such as that shown in FIG. 14( b),the change amount detection portion 71 outputs a large signal near anedge where the unsharp signal US changes, as in FIG. 14( c). Here, thevertical axis in FIG. 14( a) is the value of the image signal IS, andthe horizontal axis is the pixel position being processed. The verticalaxis in FIG. 14( b) is the value of the unsharp signal US, and thehorizontal axis is the pixel position being processed. The vertical axisin FIG. 14( c) is the value of the amount of change in the unsharpsignal US, and the horizontal axis is the pixel position beingprocessed.

The effect adjustment signal generation portion 72 adjusts the outputaccording to the amount of change that has been detected by the changeamount detection portion 71. That is, the effect adjustment signalgeneration portion 72 outputs an effect adjustment signal MOD with asmaller signal level (value) the greater the amount of this change. Forexample, as shown in FIG. 8, the signal level of the effect adjustmentsignal MOD is changed when the amount of change that is detected isequal to or greater than a predetermined value Tha, and the signal levelof the effect adjustment signal MOD is reduced in the range up to apredetermined value Thb. The signal level of the effect adjustmentsignal MOD is not changed beyond the predetermined threshold Thb. Thus,the signal level of the effect adjustment signal MOD can be changed whena steep edge region has been input, without affecting edge componentsordinarily present in natural images. Here, the horizontal axis is theamount of change, and the vertical axis is the output (signal level) ofthe effect adjustment signal MOD. It should be noted the signal level ofthe effect adjustment signal MOD that is output is output within a rangeof 0.0 to 1.0, but this may also be adjusted from 0.2 to 1.0, forinstance, depending on the strength of the visual processing. Further,the visual processing apparatus 1 is designed such that the effect ofits visual processing becomes stronger the larger the signal level ofthe effect adjustment signal MOD.

As shown in FIG. 14( d), the effect adjustment signal generation portion72 performs an output that weakens the visual effect in the flat regionEF near the edge, and for regions away from the region near the edgecreates an output for enhancing the visual effect. Using the effectadjustment signal MOD that is created by the effect adjustment signalgeneration portion 72, it is possible to achieve a visual processingapparatus 1 in which processing that weakens the visual effect iscarried out in the flat regions EF near the edge and processing thatenhances the visual effect is carried out in regions that are away fromregions near an edge. It should be noted that the vertical axis in FIG.14( d) is the value of the effect adjustment signal MOD and thehorizontal axis is the pixel position that is being processed.

As illustrated above, with the control signal generation portion 70 itis possible to detect a flat region near an edge and then create aneffect adjustment signal MOD, from the amount of change in the unsharpsignal US.

It should be noted that it is also possible for flat regions near edgesto be detected from a reduced image such as a thumbnail image in whichthe image signal has been reduced, and then output an effect adjustmentsignal MOD based on the degree of flatness near the edge or the amountof change in the unsharp signal US.

It is also possible to provide a reducing portion (not shown), forreducing the image signal, at a stage between the image signal and thecontrol signal generation portion 40, and then from the reduced imagethat is produced by the reducing portion, output an effect adjustmentsignal MOD based on the degree of flatness near the edge or the amountof change in the unsharp signal US.

By using a reduced image, it is possible to detect flat regions that arenear edges while suppressing the effects of noise. In other words, thereis less noise component in a reduced image that has been created by areduction method in which an image signal is first averaged and thendecimated, and thus by using a reduced image, it is possible to detect aflat region near an edge while suppressing the effects of noise.Further, using reduced signal allows the number of pixels to be detectedto be reduced, and this allows the number of calculations to be reduced.

It is also possible to furnish a low-pass filter, for instance, beforethe control signal generation portion 40 or the control signalgeneration portion 70, to limit the band of the image signal, and thendetect flat regions near edges. By doing this, it is possible to reducethe noise component, and flat regions near edges can be detected whilesuppressing the effects of noise.

Second Embodiment

In the first embodiment of the invention, a synthesized signal MUS thatis synthesized with different ratios of the image signal IS andsurrounding image information (unsharp signal) US according to an effectadjustment signal MOD is output, and the visual processing portion 30outputs a processed signal OS that is obtained by visually processingthe image signal IS according to the synthesized signal MUS from theeffect adjustment portion 20, but in the second embodiment of theinvention, an effect adjustment portion 21 outputs a processed signal OSthat is obtained by synthesizing a processed signal OS that has beenvisually processed and the image signal IS according to an effectadjustment signal, and this embodiment is described using FIG. 9.

FIG. 9 is a block diagram that shows the configuration of a visualprocessing apparatus 2 according to the second embodiment of theinvention. Hereinafter, sections that are identical to those of thefirst embodiment are assigned the same reference numerals as before andwill not be described in detail.

In FIG. 9, the visual processing portion 30 outputs a processed signalOS based on the image signal IS and the output US of the spatialprocessing portion 10.

The effect adjustment portion 21 interpolates the image signal IS andthe processed signal OS in accordance with the effect adjustment signalMOD in order to differ (change) the effect of visual processing. Forexample, the output OUT from the effect adjustment portion 21 iscalculated by interpolation computation like in Eq. 3 below.OUT=OS×MOD+IS×(1.0−MOD)  Eq. 3

It should be noted that Eq. 3 can be modified as shown in Eq. 4.OUT=(OS−IS)×MOD+IS  Eq. 4

Thus, according to the second embodiment of the invention, it ispossible to output a synthesized signal OUT that is synthesized varyingthe ratio of the processed signal OS and the image signal IS accordingto the effect adjustment signal MOD, so as to differ (change) the effectof visual processing.

It should be noted that it is also possible to substitute the controlsignal generation portion 70 of the first embodiment of the inventionfor the control signal generation portion 40. In this case as well,regions near an edge can be similarly detected and an effect adjustmentsignal MOD that corresponds to the amount of change in the surroundinginformation near the edge can be created.

Third Embodiment

In the first embodiment of the invention, a synthesized signal MUS thatis synthesized changing the ratio of the image signal IS and thesurrounding image information US according to an effect adjustmentsignal MOD is output, and the visual processing portion 30 outputs aprocessed signal OS that is obtained by visually processing the imagesignal according to the synthesized signal MUS from the effectadjustment portion 20, but in the third embodiment of the invention, aneffect adjustment portion 22 outputs a processed signal OS that issynthesized changing the ratio of the outputs of a visual processingportion 31 and a visual processing portion 32, whose visual processingeffects are different, in accordance with the effect adjustment signalMOD, and this embodiment is described using FIG. 10.

FIG. 10 is a block diagram that shows the configuration of a visualprocessing apparatus 3 according to the third embodiment of theinvention. Hereinafter, sections that are identical to those of thefirst embodiment are assigned the same reference numerals as before andwill not be described in detail.

The effect adjustment portion 22 synthesizes an output OSA of the visualprocessing portion 31 in which a first profile 60 has been set in theLUT and an output OSB of the visual processing portion 32 in which asecond profile 61 has been set in the LUT, whose strength of visualprocessing is different, by interpolation computation according to theeffect adjustment signal MOD that is output from the control signalgeneration portion 40, and outputs a processed signal OS. It should benoted that it is also possible to create a synthesized output byextrapolation computation. At this time, the processed signal OS is asshown in Eq. 5.OS=OSA×MOD+OSB×(1.0−MOD)  Eq. 5

It should be noted that Eq. 5 can be modified as in Eq. 6.OS=(OSA−OSB)×MOD+OSB  Eq. 6

Thus, with the third embodiment of the invention, by obtaining asynthesized output that is synthesized varying the ratio of the outputsof the visual processing portion 31 and the visual processing portion32, whose visual processing effects are different, in accordance withthe effect adjustment signal MOD, it is possible to perform visualprocessing with differing degrees of visual effect.

It should be noted that it is also possible to substitute the controlsignal generation portion 70 of the first embodiment of the inventionfor the control signal generation portion 40. In this case as well,regions near an edge can be similarly detected and an effect adjustmentsignal MOD that corresponds to the amount of change in the surroundinginformation near the edge can be created.

Fourth Embodiment

In the visual processing apparatuses of the first embodiment of theinvention through the third embodiment of the invention, a grayscaleconversion value that is based on the two-dimensional grayscaleconversion characteristics is output, but the fourth embodiment of theinvention describes a case in which grayscale conversion is performedusing a gain signal using FIGS. 11 and 12.

FIG. 11 is a block diagram that shows the configuration of a gain-typevisual processing system 4 according to the fourth embodiment of theinvention, and FIG. 12 is an explanatory diagram for describing thetwo-dimensional gain characteristics. Hereinafter, sections that areidentical to those of the first embodiment are assigned the samereference numerals as before and are not described in detail.

In FIG. 11, the gain-type visual processing system 4 is provided with again-type visual processing apparatus 5 for outputting a gain signalGAIN that is obtained by visually processing the image signal IS, and amultiplier 11 for multiplying the gain signal GAIN and the image signalIS.

The gain-type visual processing apparatus 5 is provided with the visualprocessing apparatus 1 for outputting a processing signal OS that isobtained by visually processing the image signal IS, and a divider 12for dividing the processed signal OS by the image signal IS. Here, thevisual processing apparatus 1 outputs a grayscale conversion value thatis obtained by visually processing the output of the image signal IS,and the gain-type visual processing apparatus 5 can be achieved bydividing this grayscale conversion value by the image signal IS.

The multiplier 11 multiplies the image signal IS and the gain signalGAIN that is output by the gain-type visual processing apparatus 5, andoutputs a grayscale conversion value that is produced by visuallyprocessing the output of the image signal IS.

It should be noted that it is also possible for the visual processingportion 30 to carry out processing by directly using a profile that hasthe two-dimensional gain characteristics shown in FIG. 12. Here, thevertical axis of FIG. 12 is the gain output GN, and the horizontal axisis the image signal IS. The two-dimensional gain characteristics shownin FIG. 12 are equivalent to those that are obtained by dividing theoutput of the profile of the two-dimensional grayscale characteristicsshown in FIG. 2 with the image signal IS. It is also possible to set aprofile that has these two-dimensional gain characteristics in the LUTof the visual processing portion 30 of the visual processing apparatus1. If a profile of the two-dimensional gain characteristics is set inthe LUT of the visual processing portion 30 in advance, then the gainsignal GN and the gain signal GAIN become equal, and thus it is possibleto achieve the gain-type visual processing apparatus 5 without thedivider 12.

With the gain-type visual processing apparatus 5, there is little changein the processed signal with respect to the change in the input imagesignal IS, and thus it is possible to reduce the bit number of the inputsignal and also to reduce the circuit scale. Additionally, if the visualprocessing portion 30 is provided with a 2D LUT, then it is possible toreduce the memory capacity.

Thus, with the gain-type visual processing system 4 of the fourthembodiment of the invention, saturation of the grayscale can besuppressed easily and excellent visual processing can be achieved bycontrolling the gain signal GAIN.

It should be noted that the visual processing apparatus 1 of the firstembodiment of the invention can be replaced with the visual processingapparatus 2 of the second embodiment of the invention. In this case aswell, the gain-type visual processing apparatus 5 can be similarlyachieved.

The visual processing apparatus 1 of the first embodiment of theinvention can be replaced with the visual processing apparatus 3 of thethird embodiment of the invention. The gain-type visual processingapparatus 5 can be similarly achieved in this case as well.

Thus, according to the first embodiment of the invention through thefourth embodiment of the invention, it is possible to realize visualprocessing in which side effects are suppressed, even when an image withsteep edge regions has been input.

It should be noted that the visual processing apparatus described inthis embodiment can be provided in or connected to a device for handlingmoving images, and may create an effect adjustment signal MOD from theimage of each frame or each field. The control signal generation portion40 can extract edge information or flatness information from a frameimage at least one (frame) prior when the image signal is a frame image,or from a field image at least one (field) prior when the image signalis a field image. By doing this, the visual processing apparatus can usean effect adjustment signal MOD that corresponds to the edge informationor the flatness information from the top of the frame. It is alsopossible for the visual processing apparatus to extract edge informationor flatness information from a field image at least one (field) prior,and use an effect adjustment signal MOD that corresponds to the edgeinformation or the flatness information from the top of the field image.It is also possible for the control signal generation portion 40 toextract edge information or flatness information from a frame image atleast one (frame) prior or from a field image at least one (field)prior, and by doing so it becomes easy to coordinate the delay of thecircuit and the circuit scale can be reduced.

It should be noted that the various functions, such as the spatialprocessing function, the effect adjustment function, and the visualprocessing function, of the visual processing apparatuses or visualprocessing system of the first embodiment of the invention through thefourth embodiment of the invention can be achieved by hardware that usesan integrated circuit, for example, or may also be implemented bysoftware that operates using a central processing unit (CPU) or adigital signal processing apparatus or the like.

First, if the various functions are implemented by hardware, then thevarious functions of the embodiments of the invention can be provided onseparate integrated circuits, or alternatively some or all of these maybe integrated into a single chip.

The integrated circuit may be realized by a dedicated circuit or ageneral purpose processor. For example, after processing thesemiconductor chip, it is possible to employ a programmable FPGA (FieldProgrammable Gate Array) or a reconfigurable processor in which theconnections and settings of the cells within the integrated circuit canbe reconfigured.

Further, if other technologies that improve upon or are derived fromsemiconductor technologies introduce integration technology thatsupplants integrated circuit technology, then naturally it is alsopossible to integrate the functional blocks using that technology.Advances in biotechnology may lead to biocomputer applications.

Next, a case in which the various functions are realized by software isdescribed using FIG. 15. FIG. 15 is a block diagram showing theconfiguration of a computer according to an embodiment of the invention.

In FIG. 15, a computer 6 is provided with a CPU 100 that executes thecommands of the various programs, a read-only memory 101 in which theprograms are stored (hereinafter, referred to as “ROM 101”), arandom-access memory 102 for holding temporarily stored data (hereafter,referred to as “RAM 102”), an input portion 103 for inputting an image,an output portion 104 for outputting an image, and a memory portion 105for storing programs and various types of data.

It can also be provided with a communications portion 106 forcommunicating with the outside, and a drive 107 for appropriatelyconnecting to an information storage medium.

The various functional portions send and receive control signals anddata, for example, over a bus 110.

The CPU 100 executes the processing of the various functions inaccordance with programs that are stored in the ROM 101.

The ROM 101 stores programs and profiles, for example.

The RAM 102 temporarily stores the data necessary for the processing ofthe various functions by the CPU 100.

The input portion 103 inputs images. For example, it receiveselectromagnetic waves and decodes the received signal that is receivedto produce a video signal. It may also obtain digital images directlyvia a land line.

The output portion 104 outputs images. For example, it outputs to adisplay device such as a liquid crystal display device or a plasmadisplay device.

The memory portion 105 is made from a magnetic memory or the like, andstores various programs and data.

The communications portion 106 is connected to a network 111, and viathe network 111 it obtains programs and, as necessary, installs theprograms that it has obtained onto the memory portion 105. Thus, thecomputer 6 can download programs through the communications portion 106.

The drive 107 appropriately connects with an information storage medium,and obtains storage information that is stored on the informationstorage medium. The information storage medium is a disk 108 such as amagnetic disk, magneto-optical disk, or optical disk, or a memory card109 such as a semiconductor memory.

It should be noted that it is also possible to store the profiles orprogram for executing the various functions on the disk 108 or a memorycard 109 such as a semiconductor memory, and to provide that informationto the computer 6.

It is also possible to provide the program to the computer in advancebundled with dedicated hardware, or for the program to be bundled in theROM 101 or the storage portion 105 in advance.

The program can be adopted by devices that handle images, such asinformation processing devices, televisions, digital cameras, portabletelephones, and PDAs. The program can be installed in or connected to adevice that handles images, and executes visual processing in which sideeffects are suppressed in flat regions that are near edges.

It should be noted that the visual processing apparatus can be providedin or connected to a device for handling moving images, and may createan effect adjustment signal MOD from the image of each frame or eachfield. The control signal generation portion 40 can extract edgeinformation or flatness information from a frame image at least one(frame) prior when the image signal is a frame image, or from a fieldimage at least one (field) prior when the image signal is a field image.By doing this, the visual processing apparatus can use an effectadjustment signal MOD that corresponds to the edge information or theflatness information from the top of the frame. It is also possible forthe visual processing apparatus 1 to extract edge information orflatness information from a field image at least one (field) prior, soas to use an effect adjustment signal MOD that corresponds to the edgeinformation or the flatness information from the top of the field image.It is also possible for the control signal generation portion 40 toextract edge information or flatness information from a frame image atleast one (frame) prior or from a field image at least one (field)prior, and by doing so it becomes easy to coordinate the delay of thecircuit and the circuit scale can be reduced.

With the visual processing apparatus, visual processing method, program,recording medium, display device, and integrated circuit according tothe invention, it is possible to visually process the image signal, andin particular, it is possible to inhibit side effects even when an imagethat has sharp edge regions has been input, and thus these are usefullyin fields relating to video and image processing, and the visualprocessing apparatus, visual processing method, program, recordingmedium, display device, and integrated circuit according to theinvention can be implemented in these fields.

1. A visual processing device, comprising: a spatial processing portionoperable to output a processed signal generated by performingpredetermined spatial processing on an input image signal based oninformation from a plurality of pixels surrounding a target pixel of theinput image signal; and a visual processing portion operable to performvisual processing using the input image signal and the processed signal,and operable to output an output signal as a result of the visualprocessing, the visual processing being performed using the input imagesignal and the processed signal output by the spatial processingportion, so as to enhance a local contrast of the input image signal,wherein the visual processing portion reduces an effect of theenhancement of the local contrast with respect to a flat region in theinput image signal, the flat region being proximal to a detected edge ofthe input image signal.
 2. The visual processing device according toclaim 1, wherein the flat region, which is proximal to the detected edgeof the input image signal, is determined by detecting an edge of anunsharp signal of the input image signal.
 3. The visual processingdevice according to claim 2, wherein the unsharp signal is the processedsignal output by the spatial processing portion.
 4. An image displaydevice, comprising: a spatial processing portion operable to output aprocessed signal generated by performing predetermined spatialprocessing on an input image signal based on information from aplurality of pixels surrounding a target pixel of the input imagesignal; a visual processing portion operable to perform visualprocessing using the input image signal and the processed signal, andoperable to output an output signal as a result of the visualprocessing, the visual processing being performed using the input imagesignal and the processed signal output by the spatial processingportion, so as to enhance a local contrast of the input image signal;and a display portion operable to display the output signal, wherein thevisual processing portion reduces an effect of the enhancement of thelocal contrast with respect to a flat region in the input image signal,the flat region being proximal to a detected edge of the input imagesignal.
 5. A television device comprising: a reception portion operableto receive a video signal; a decoding portion operable to decode thevideo signal and output an image signal; a spatial processing portionoperable to output a processed signal generated by performingpredetermined spatial processing on the image signal based oninformation from a plurality of pixels surrounding a target pixel of theimage signal; a visual processing portion operable to perform visualprocessing using the image signal and the processed signal, and operableto output an output signal as a result of the visual processing, thevisual processing being performed using the image signal and theprocessed signal output by the spatial processing portion, so as toenhance a local contrast of the image signal; and a display portionoperable to display the output signal, wherein the visual processingportion reduces an effect of the enhancement of the local contrast withrespect to a flat region in the image signal, the flat region beingproximal to a detected edge of the image signal.
 6. A portableinformation terminal device, comprising: a reception portion operable toreceive a video signal; a decoding portion operable to decode the videosignal and output an image signal; a spatial processing portion operableto output a processed signal generated by performing predeterminedspatial processing on the image signal based on information from aplurality of pixels surrounding a target pixel of the image signal; avisual processing portion operable to perform visual processing usingthe image signal and the processed signal, and operable to output anoutput signal as a result of the visual processing, the visualprocessing being performed using the image signal and the processedsignal output by the spatial processing portion, so as to enhance alocal contrast of the image signal; and a display portion operable todisplay the output signal, wherein the visual processing portion reducesan effect of the enhancement of the local contrast with respect to aflat region in the image signal, the flat region being proximal to adetected edge of the image signal.
 7. A camera, comprising: an imagingportion operable to capture an image and generate an image signal; aspatial processing portion operable to output a processed signalgenerated by performing predetermined spatial processing on the imagesignal based on information from a plurality of pixels surrounding atarget pixel of the image signal; a visual processing portion operableto perform visual processing using the image signal and the processedsignal, and operable to output an output signal as a result of thevisual processing, the visual processing being performed using the imagesignal and the processed signal output by the spatial processingportion, so as to enhance a local contrast of the image signal; and adisplay portion operable to display the output signal, wherein thevisual processing portion reduces an effect of the enhancement of thelocal contrast with respect to a flat region in the input image signal,the flat region being proximal to a detected edge of the image signal.8. A visual processing method, comprising: a spatial processing step ofoutputting a processed signal generated by performing predeterminedspatial processing on an input image signal based on information from aplurality of pixels surrounding a target pixel of the input imagesignal; and a visual processing step of performing visual processingusing the input image signal and the processed signal, and outputting anoutput signal as a result of the visual processing, the visualprocessing being performed using the input image signal and theprocessed signal output by the spatial processing step, so as to enhancea local contrast of the input image signal, wherein the visualprocessing step includes reducing an effect of the enhancement of thelocal contrast with respect to a flat region in the input image signal,the flat region being proximal to a detected edge of the input imagesignal.
 9. An integrated circuit used for an image output device,executing: a spatial processing step of outputting a processed signalgenerated by performing predetermined spatial processing on an inputimage signal based on information from a plurality of pixels surroundinga target pixel of the input image signal; and a visual processing stepof performing visual processing using the input image signal and theprocessed signal, and outputting an output signal as a result of thevisual processing, the visual processing being performed using the inputimage signal and the processed signal output by the spatial processingstep, so as to enhance a local contrast of the input image signal,wherein the visual processing step includes reducing an effect of theenhancement of the local contrast with respect to a flat region in theinput image signal, the flat region being proximal to a detected edge ofthe input image signal.
 10. A non-transitory computer-readable storagemedium having an image processing program stored thereon, the imageprocessing program for performing a visual processing by a computer, andthe image processing program causing the computer to execute a visualprocessing method comprising: a spatial processing step of outputting aprocessed signal generated by performing predetermined spatialprocessing on an input image signal based on information from aplurality of pixels surrounding a target pixel of the input imagesignal; and a visual processing step of performing visual processingusing the input image signal and the processed signal, and outputting anoutput signal as a result of the visual processing, the visualprocessing being performed using the input image signal and theprocessed signal output by the spatial processing step, so as to enhancea local contrast of the input image signal, wherein the visualprocessing step includes reducing an effect of the enhancement of thelocal contrast with respect to a flat region in the input image signal,the flat region being proximal to a detected edge of the input imagesignal.